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Specification 

1. Title of the Invention 

CHARACTER/GRAPHIC SEPARATING SYSTEM FOR IMAGE INFORMATION 

2 . Claim for the Patent 

A character/graphic separating system which separates image 
information including mixed characters and graphics into a 
character area and a graphic area, comprising at least: 

a first step of extracting a closed area by tracking a 
boundary of black pixels included in the image information; 

a second step of separating the character area and the 
graphic area depending on a size of the closed area; 

a third step of tracking an inner boundary of the separated 
graphic area, and extracting an internal area; and 

a fourth step of detecting whether or not the extracted 
internal area includes the black pixel, wherein 

the first step to the fourth step are repeated until the 
black pixels are not detected in the internal area, thereby 
hierarchically separating the character area and the graphic 
area. 



3 . Detailed Description of the Invention 
(Industrial Application Field) 

The present invention relates to a mixed mode communication 
of exchanging a document including mixed characters and graphics, 
and more specifically to a character/graphic separating system 
of automatically separating the characters and graphics in the 
image information. 
(Conventional Art) 

A facsimile communication of transmitting a document by 
sequentially discriminating the black and white pixels from the 
start of the document has been commonly used as means for 
transmitting a document including mixed characters and graphics. 
However, since data is processed in a pixel unit regardless of 
characters or graphics in the facsimile communication, there has 
been a problem of poor transmission efficiency with the 
facsimile communication. 

As an improved communication system from the facsimile 
communication, there is a mixed mode communication recently 
developed and attracting attention as a system of separating 
characters and graphics, and coding the characters by a 
character code and the graphics in the conventional pixel unit. 
Since a separated character area is coded by a character code in 
the mixed mode communication, the transmission efficiency has 
been largely improved as compared with the conventional system 
of coding data in a pixel unit, and a document can be edited 
with a character changed or the position of a character changed 
on the receiver side. 



Figure 1 shows a document including mixed characters and 
graphics, and the process of separating the characters and the 
graphics of the document in the conventional mixed mode 
communication is described below. 

(1) For the binary document image of black and white pixels 
including characters and graphics, data is scanned in the "row" 
direction (x direction) in, for example, a run length smoothing 
algorithm, white pixels are converted into black pixels when the 
run lengths of adjacent white pixels are equal to or lower than 
a predetermined threshold, and the black pixels are identified 
as black pixels as is. 

(2) The data is processed similarly in the "column" 
difference (y direction) . 

(3) The black pixels obtained as the results of the 
processes in the row and column directions are "ANDed" and a 
black area is determined. 

(4) Furthermore, the size of each closed area as a black 
area is determined based on a predetermined criterion, and the 
character and graphic areas are separated by determining the 
closed area as the character area (area A) and the graphic area 
(area B) based on the predetermined criterion. 

The separated character and graphic areas are transmitted by 
performing an appropriate coding process. 
(Problems to be Solved by the Invention) 

However, the conventional character/graphic separating 
system separates a character and graphic areas only in a two- 
dimensional method without considering a hierarchical structure, 
and it is difficult to identify a character in the area B 
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although there is a character still included in the graphic area 
(area B) shown in Figure 1. 

Especially, when graphics are configured mostly by 
characters such as the "table", the entire "table" is determined 
as a graphic area in the conventional character/graphic 
separating system, and is processed in a pixel (dot) unit. 
Therefore, as compared with the method of coding data by a 
character code of a character area, a large amount of data is to 
be coded in a pixel unit with poor transmission efficiency, 
thereby incurring the reduction in transmission efficiency and 
an impossible document edition of characters displayed in a 
pixel unit. 

Therefore, in a document including mixed characters and 
graphics, there has been an earnest request for a system of 
separating character and graphic areas as correctly and 
efficiently as possible, but there has no such systems disclosed. 
(Means for Solving the Problems) 

The present invention has been developed to solve the above- 
mentioned problems with the conventional technology, and aims at 
providing a character/graphic separating system for image 
information capable of efficiently and correctly separating a 
character area and a graphic area in a document including mixed 
characters and graphics. 

The feature of the present invention resides in extracting a 
closed area from input image information in which a document 
including mixed characters and graphics is converted into a 
binary image by tracking the boundary of a black area in the 
image, analyzing (hereinafter referred to as a "topology 
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analysis") the inclusive relation of each of the extracted 
closed areas, describing it as a hierarchical tree structure 
(hereinafter referred to as a "topological structure"), and 
converting the topological structure into a document structure 
in which characters and graphics are separated so as to be 
adapted to the communication system. 

The present invention is described below in detail with 
reference to the attached drawings . 
(Configuration and Operation of the Invention) 

Figure 2 is a block diagram of the procedure of a system of 
hierarchically separating characters and graphics according to 
the present invention. A document 1 which includes mixed 
characters and graphics is converted into binary black and white 
data as binary black and white image information by a scanner 
etc., a closed area is extracted by tracking the boundary of a 
black area, a topology analysis 2 of analyzing an internal 
structure of the extracted closed area is performed, and a 
topological structure 4 describing a hierarchical relationship 
of each closed area is generated. Furthermore, a structure 
conversion 5 is performed to easily coding data by separating 
the topological structure 4 into characters and graphics, and a 
final document structure is generated, thereby completely 
separating the characters and graphics. 

Mainly described below with reference to an example of an 
input image shown in Figure 1 is the procedure of a topology 
analysis 3 and the topological structure 4 as the features of 
the present invention. 



(1) Topology Analysis 

The procedure of the topology analysis 3 is described below. 

1) A black pixel as a start point is found by a raster 
scanning process, a boundary of 8 pixels are tracked, and a 
closed area is extracted. At this time, each boundary coordinate 
is stored. 

2) A media (characters and graphics) are analyzed in the 
extracted closed area. Figure 3 shows a criterion in determining 
characters and graphics (for an alphabetical document) . 
Depending on the size of a rectangle enclosing the extracted 
closed area, the portion with diagonal lines shown in Figure 3 
is defined as a character, and other portions are defined as 
graphics. In this determining criterion, the horizontal contact 
of a character is also considered, and the thresholds shown in 
Figure 3 are w x for the maximum width of a target character, h x 
for the maximum height of a target character, and h n for the 
minimum height of a target character. 

3) When data is determined as a character, the character is 
sequentially written to the area corresponding to a character 
node . 

4) When data is determined as graphics, a child node having 
its closed area is generated. When there is a hole as the 
example shown in Figure 1, a grandchild node having an area 
inside the inner boundary is generated. An area can be extracted 
in the method shown in Figure 4. That is, as compared with the 
target closed area as shown by (a) of Figure 4, the area inside 
the outer boundary is regarded as a set of lines enclosed by the 
two points (x ia , y-j) and (x ie , y-j) on the boundary stored by 
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tracking the boundary. Thus, by sequentially copying the pixel 
value of each coordinate of the line enclosed by the two points 
for each line, the image shown by (b) of Figure 4 is obtained. 
Simultaneously, when each pixel value is inverted and 
sequentially copied for each line, the inverse video as shown by 
(c) of Figure 4 is obtained. The outer boundary shown in (c) of 
Figure 4 corresponds to the inner boundary of (a) of Figure 4. 
Furthermore, by performing on the image of (c) of Figure 4 the 
same process as the process for obtaining (b) and (c) of Figure 
4 from (a) of Figure 4, (d) and (e) of Figure 4 are obtained. 
However, (e) of Figure 4 is a practical result of extracting the 
area inside the inner boundary in (a) of Figure 4 . 

5) If no black pixels are included in the area inside the 
inner boundary obtained in 4) , the process terminates. If they 
are included, 1) to 4) are repeated using the grandchild node as 
a parent. 

(2) Topological Structure 

Figure 5 shows a topological structure obtained as a result 
of performing the above-mentioned process on the image shown in 
Figure 1. 

In Figure 1, the node at the first level indicates the 
entire image shown in Figure 1, and the nodes at the second 
level are a character node indicating a character area and a 
child node indicating a graphic area. Since the graphic area 
corresponding to the child node has a deep structure, the 
grandchild nodes 1 and 2 corresponding to the internal area 
extracted by the above-mentioned algorithm are generated as the 
third level. Furthermore, the processes are continuously 
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performed on the image of the area corresponding to the 
grandchild nodes, a character node is added below the grandchild 
node 1, and a character node and a child node indicating 
graphics are added as the fourth level below the grandchild node 
2 . Since there is no deep structure in the graphic area 
corresponding to the node at the fourth level, the process 
terminates here. 

As described above, the structuring process to 
hierarchically separate a character area and a graphic area 
according to the gist of the present invention terminates. 
However, when the mixed mode communication is performed, it is 
necessary to follow the layout structure (document structure) 
including a page (p) , a frame (F) , and a block (B) regulated in 
the CCITT Recommendation T.73. 

Therefore, described below is the procedure of converting 
the topological structure according to the present invention to 
the document structure regulated in the CCITT Recommendation 
T. 73 . 

(3) Structure Conversion and Document Structure 

1) Define the node of the root (first level) as a page, and 
a node positioned at an odd level and having no lower node is 
erased. 

2) A node at an even level is defined as a block when it has 
no lower node. Otherwise, it is defined as a frame, and a block 
having as contents a result of obtaining an exclusive logical 
sum of a corresponding area and all nodes at one level lower is 
added to the level below the corresponding frame. 
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3) The node at an odd level is defined as a frame when the 
number of lower nodes is two or more, and erased when the number 
is one and the lower node is added below the upper node. 

4) The top-down process 2) to 3) above is repeated on a 
structure tree. Figure 6 shows the result of converting the 
topological structure shown in Figure 5 to the layout structure 
in the procedure above. The node at the first level shown in 
Figure 5 is a page in Figure 6 by 1) above. Next, as for the 
node at the second level shown in Figure 5, the character node 
is determined as a block as is and the node having a graphic 
area is determined as a frame by 2) above in Figure 6. 
Simultaneously, the block having as the contents a result of 
obtaining an exclusive logical sum of the graphic area and the 
area of the two nodes (grandchild nodes 1 and 2) at the third 
level is added below the frame. Furthermore, the grandchild node 
1 at the third level shown in Figure 5 is erased by 3) above, 
and the lower character node at the fourth level is added as a 
block below the above-mentioned frame. The grandchild node 2 is 
determined as a frame by 3) above. Finally, the character node 
at the fourth level below the grandchild node 2 and the child 
node are added as blocks by 2) above. 

As described above, according to the present invention, the 
characters included in the graphic area can be efficiently and 
correctly separated by hierarchically separating the graphic 
area. 

Described next is the configuration of the apparatus for 
hierarchically separating data. 



Figure 7 shows the outline of the character/graphic 
separating system according to the present invention. An input 
unit 7 binarizes the image information input from a scanner (not 
shown in the attached drawings) at any level, and stores the 
binary information in built-in memory. A structure analysis unit 
8 obtains structured data by analyzing the structure of an image 
as the feature of the present invention, and determining the 
media (characters and graphics) . A character recognition unit 9 
determines whether or not the character area separated by the 
structure analysis unit 8 is correct, and converts the character 
area into a character code when it is correct. An accumulation 
unit 10 accumulates structured data in a disk etc., and the 
output of the accumulation unit 10 is provided for printing, 
editing, and transmission. 

The character recognition unit 9 is an existing technique, 
and is known by a publication "Introduction to Character 
Recognition" edited by Hashimoto, published by the Institute of 
Electronics, printed by Ohmsha, issued in March, 1982, etc. 
Therefore, the structure analysis unit 8 is described below in 
detail. 

Figure 8 is a block diagram of the structure analysis unit 8. 
A boundary tracking unit 11 tracks a boundary of 8 pixels for a 
binary black and white pixel image, and stores boundary 
coordinate values. A character/graphics separation unit 12 
determines characters and graphics depending on the size of a 
closed area extracted by the boundary tracking unit 11. Data 
storage units 13 and 14 store data in the character or graphic 
areas. An internal area extraction unit 15 extracts an internal 
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area of the graphic area determined by the character/graphics 
separation unit 12. An internal area extraction unit 15 extracts 
an internal area of a graphic area determined by the 
character /graphics separation unit 12. A termination 
determination unit 16 determines whether or not there is a black 
pixel in all extracted internal areas. A gate circuit 17 
transmits the data stored in the data storage units 13 and 14 to 
the converter unit 18 when the termination determination unit 16 
determines that no black pixels are detected in all internal 
areas and considers that the topology analysis has been 
terminated. A converter unit 18 converts a topological structure 
into a document structure (layout structure) . 

That is, the structure analysis unit 8 generates a closed 
area by the boundary tracking unit 11, and determines the closed 
area a character area or a graphic area depending on the size of 
the closed area. The character /graphics separation unit 12 
separates the areas, checks whether or not a closed area is 
included in the separated graphic area. The internal area 
extraction unit 15 extracts the area. If a closed area is 
included in the graphic area, the boundary tracking unit 11 and 
the character/graphics separation unit 12 repeatedly separate 
the characters and the graphics as a structuring process as 
described above. Finally, the character areas and the graphic 
areas can be correctly separated. 

In the description above, one document includes a graphic 
area for example. However, when a plurality of graphic areas are 
included, an internal area is sequentially extracted using one 
15 and 16 for each graphic area, or a plurality of internal area 
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extraction units 15 and termination determination units 16 are 
provided so that the gate circuit 17 can be operated when no 
black pixel is detected in all closed areas. 

Although an example of converting a topological structure 
into a document structure according to the CCITT Recommendation 
is described above, the present invention is not limited to this 
application, and the structure can be converted into any other 
document structures . 
(Advantages of the Invention) 

As described above, the present invention can extract a 
closed area by tracking a boundary of a black area in an image, 
and repeatedly analyzes the internal structure of the extracted 
closed area, thereby efficiently and correctly separating a 
character area and a graphic area. Therefore, the subsequent 
document edition can be easily performed, an mixed mode 
communication can be performed with high transmission efficiency 
and accumulation efficiency by appropriate coding data for each 
of the separated areas, thereby greatly enhancing the effect of 
the invention in reducing a necessary amount of image 
information and improving a transmission speed. 

4. Brief Description of the Drawings 

Figure 1 shows an example of a document having an internal 
structure for description of the conventional character/graphic 
separating system. Figure 2 is a block diagram of the flow of 
the character/graphic separating system according to the present 
invention. Figure 3 shows the criterion of characters and 
graphics according to the present invention. Figure 4 is an 
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explanatory view of the method of extracting an internal 
structure of a closed area according to the present invention. 
Figure 5 shows the topological structure according to the 
present invention. Figure 6 shows the conversion from a 
topological structure to a document structure according to the 
present invention. Figure 7 shows the outline of the 
character/graphic separating method according to the present 
invention. Figure 8 is a block diagram of the structure analysis 
unit according to the present invention. 

7 ... input unit, 

8 ... structure analysis unit, 

9 ... character recognition unit, 

10 ... accumulation unit, 

11 ... boundary tracking unit, 

12 ... character/graphics separation unit, 
13, 14 ... data storage unit, 

15 ... internal area extraction unit, 

16 ... termination determination unit, 

17 ... gate circuit, 

18 ... converter unit. 
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Figure 1 
#1 AREA A 
#2 AREA B 

Figure 2 

1 DOCUMENT 

2 CONVERSION INTO BINARY BLACK AND WHITE DATA 

3 TOPOLOGY ANALYSIS 

4 TOPOLOGICAL STRUCTURE 

5 STRUCTURE CONVERSION 

6 DOCUMENT STRUCTURE 

Figure 3 

#1 HEIGHT OF EXTRACTED AREA 

#2 GRAPHICS 

#3 CHARACTER 

#4 MAXIMUM WIDTH OF TARGET CHARACTER 

#5 MAXIMUM HEIGHT OF TARGET CHARACTER 

#6 MINIMUM HEIGHT OF TARGET CHARACTER 

#7 WIDTH OF EXTRACTED AREA 

Figure 4 

#1 OUTER BOUNDARY 
#2 INNER BOUNDARY 

#3 AREA WITH DIAGONAL LINES: BLACK PIXELS 
#4 COPY OF PIXELS IN AREA INSIDE THE INNER BOUNDARY 
#5 INVERT AND COPY PIXEL VALUE IN AREA INSIDE THE OUTER 
BOUNDARY 
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Figure 5 

#1 PARENT NODE 

#2 CHARACTER NODE 

#3 CHILD NODE 

#4 GRANDCHILD NODE 

#5 FIRST LEVEL 

#6 SECOND LEVEL 

#7 THIRD LEVEL 

#8 FOURTH LEVEL 

Figure 6 
#1 PAGE 
#2 FRAME 
#3 BLOCK 

Figure 7 
#1 INPUT 

7 INPUT UNIT 

8 STRUCTURE ANALYSIS UNIT 

9 CHARACTER RECOGNITION UNIT 

10 ACCUMULATION UNIT 

Figure 8 
#1 INPUT 

9 CHARACTER RECOGNITION UNIT 

11 BOUNDARY TRACKING UNIT 

12 CHARACTER/GRAPHICS SEPARATION UNIT 
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13 DATA STORAGE UNIT 

14 DATA STORAGE UNIT 

17 GATE CIRCUIT 

18 CONVERTER UNIT 

15 INTERNAL AREA EXTRACTION UNIT 

16 TERMINATION DETERMINATION UNIT 
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